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Self-Organizing  Neural  Circuits  for  Sensory-Guided  Motor  Control 

Co-PIs:  Daniel  Bullock  &  Stephen  Grossberg 

Overview.  The  goal  of  this  project  was  to  understand  how  the  flexibility  and  accuracy  of 
human  movement  are  achieved  by  self-organizing  neural  circuits  that  operate  under 
continuous  or  intermittent  sensory  guidance.  To  that  end,  we  studied  separate  neural 
circuits  that  function  as  movement  generators  for  visually  guided  arm  movements,  and 
for  visually  guided  saccadic  eye  movements.  In  each  case,  the  goal  was  to  specify 
mathematical  models  of  circuit  operations  that  enable  movements  to  be  generated,  to  be 
adapted  on  the  fly  to  novel  contingencies,  and  to  be  adapted  over  several  learning  trials  to 
recurring  contingencies.  Each  model  was  developed  to  satisfy  multiple  constraints. 

These  constraints  were  of  three  types.  First  were  behavioral  constraints.  The  proposed 
circuits  had  to  have  operating  characteristics  that  matched  the  operating  characteristics  of 
measured  human  performance  and  learning,  such  as  the  ability  to  recover  from  damage  to 
one  part  of  a  circuit.  Second  were  design  constraints.  The  proposed  circuits  had  to 
exhibit  stability,  robustness,  short-term  flexibility,  and  long-term  adaptability  as 
fundamental  properties.  Third  were  neurobiological  constraints.  The  proposed  circuits 
had  to  be  consistent  with,  and  provide  an  integrative  explanation  of,  many 
neuroanatomical,  neurophysiological,  and  biophysical  observations.  Circuit  models  that 
satisfy  all  three  sets  of  criteria  make  contributions  to  fundamental  behavioral  and  neural 
sciences,  while  also  constituting  prototypes  of  high-competence  adaptive  systems  that 
can  be  developed  for  many  technological  applications  in  adaptive  robotics.  To  ensure 
that  the  latter  goal  was  facilitated,  all  circuit  models  were  published  with  a  complete 
mathematical  specification,  typically  a  system  of  ordinary  differential  equations.  We 
now  summarize  the  major  results. 

1 .  The  Vector  Integration  To  Endpoint  (VITE)  model  for  generation  of  voluntary  point- 
to-point  reaching  movements.  Relevant  reference  numbers  (see  attached  publication 
list):  1, 3, 21, 22, 23, 29, 30, 36, 43,  78.  Boldface  marks  recent  major  publications. 

The  initial  version  of  this  neural  circuit  model  was  developed  in  1985-86  as  a  central 
pattern  generator  (CPG)  for  voluntary  point-to-point  movements  of  the  arm.  Such 
voluntary  movements  can  be  started  and  stopped,  or  slowed  down  and  sped  up,  at  will. 
Normally  such  movements  exhibit  smooth,  bell-shaped  velocity  profiles  that  start  at  zero 
velocity  and  return  to  zero  velocity  after  passing  through  a  single  peak  velocity  whose 
value  scales  with  movement  distance.  The  VITE  circuit  model  explains  the  origin  of 
these  smooth,  unimodal  velocity  profiles.  The  model  also  incorporates  a  mechanism  for 
voluntary  scaling  of  movement  speed,  and  thus  movement  duration.  Thus  the  model 
describes  a  way  that  one  component  of  what  has  been  called  the  will  can  be  realized  as  a 
mechanism  for  starting,  stopping,  and  modulating  the  execution  of  movement  plans.  The 
VITE  model  has  served  as  a  basis  for  developing  a  more  comprehensive  theory  of 
trajectory  generation  for  point-to-point  and  more  complex  movements.  The  model’s 
name,  VITE,  pronounced  VEET,  is  an  acronym  inspired  by  the  French  word,  vite,  for 
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“quickly”,  because  the  circuit  allows  continuous  variation  of  movement  speed  while 
overall  velocity  profile  shape  and  movement  vector  are  preserved. 

The  VITE  model  has  recently  been  elaborated  to  explain  how  arm  movements  of  variable 
speed  can  be  planned  and  made  under  conditions  of  unexpected  load,  including  the 
presence  of  obstacles  (Bullock,  Cisek  &  Grossberg,  1998;  Cisek,  Grossberg  &  Bullock, 

1 998).  Also  modeled  were  reciprocal  links  between  the  cortical  VITE  circuit  and  the 
spinal  cord.  This  work  shows  how  sensory  feedback  can  improve  force  regulation  under 
a  variety  of  scenarios  that  arise  during  movement  under  load  and  during  relaxation.  This 
more  mature  model  can  simulate  and  explain  the  movement-related  activation  profiles  of 
6  well-established  cell  types  in  motor  and  parietal  cortex  during  a  variety  of  movement 
tasks.  The  model  also  allows  strong  predictions  to  be  made  regarding  the  form  of  the 
movement-related  activation  profiles  that  should  be  observed  under  novel  experimental 
protocols.  No  other  extant  model  can  make  predictions  regarding  such  a  range  of  cell 
types  in  sensory-motor  cortex,  because  other  models  have  failed  to  specify  the  closed 
loop  through  which  cortical  trajectory  generation  and  sensory  feedback  work  together. 

From  a  control  theoretic  or  robotics  perspective,  the  latest  circuit  includes  a  speed- 
scalable  trajectory  generator  in  series  with  a  PID  controller.  However,  the  model 
improves  upon  such  a  standard  in-series  control  design  in  several  ways,  as  detailed  in 
Bullock,  Cisek  &  Grossberg  (1998).  Briefly,  routing  the  feedback  to  the  trajectory 
generator  as  well  as  to  the  PID  nodes  allows  the  system  to  adjust  the  desired  trajectory  to 
loading  conditions  and  thus  to  avoid  the  high  energy  costs  and  instability  that  would  be 
entailed  by  attempting  to  move  too  large  a  load  too  rapidly. 

2.  The  Factorization  of  LEngth  and  TEnsion  (FLETE)  model  for  independent  control  of 
joint  position  and  joint  stiffness  by  an  opponent  muscle  system.  Relevant  recent 
reference  numbers:  16, 17, 24,  25, 26, 43, 47,  86. 

Muscles  are  wonderful  actuators,  with  excellent  mass  distribution  properties,  high  force 
to  weight  ratios,  and  intrinsic  damping.  Understanding  how  they  are  controlled  in  a  goal- 
directed  way  benefits  basic  science,  medical  science,  and  robotics  science.  The  initial 
version  of  the  FLETE  model  for  muscle  control  was  developed  to  explain  how  circuits  in 
the  spinal  cord  allow  higher  brain  circuits  to  use  opposing  muscle  pairs  to  simultaneously 
control  joint  angle  and  joint  stiffness  (Bullock  and  Grossberg,  1991).  This  is  a 
fundamental  property  because  it  allows  animals  to  freely  retune  themselves  along  a 
continuum  from  rigid  to  fully  compliant  systems.  The  FLETE  model  explains  how  this 
retuning  is  mediated  by  the  inhibitory  and  excitatory  interactions  among  distinct  cell 
types  associated  with  control  of  opposing  muscles.  The  FLETE  theory  explains  how 
these  interactions  work  to  maintain  an  appropriate  balance  of  forces  across  the  joint 
despite  a  number  of  intrinsic  non-linearities  in  the  system.  This  circuit  model  highlights 
the  fact  that  many  non-linearities  can  be  compensated  by  the  opponent  spinal  circuit  even 
in  the  absence  of  any  feedback  from  kinesthetic  and  force  sensors  embedded  in  muscles 
and  joints.  Thus,  although  it  delineates  the  distinct  benefits  of  several  types  of  sensory 
feedback,  the  model  also  helps  explain  how  animals  are  able  to  continue  to  make  fairly 
accurate  single-joint  movements  even  when  deprived  of  all  sources  of  sensory  feedback. 
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The  model  thus  explicates  how  careful  circuit  design  (by  evolution)  has  led  to  great 
robustness  to  loss  of  normal  sources  of  feedback.  Further  development  of  the  model  has 
shown  how  to  understand  the  spinal  circuits  of  mammals  as  a  cumulative  solution  to  a  set 
of  inherent  challenges  associated  with  the  use  of  muscle-like  actuation.  As  such,  the 
model  provides  insight  into  the  degrees  of  freedom  and  design  principles  that  will  need  to 
be  used  by  roboticists  when  it  finally  becomes  possible  to  manufacture  and  deploy 
muscle-like  actuators.  A  1998  study  (#86)  showed  how  the  FLETE  circuit  parameters 
can  be  self-organized. 

Among  the  central  theses  of  the  FLETE  model  is  that  the  Renshaw  cells  play  several 
critical  roles  in  addition  to  those  proposed  by  physiologists  who  attempted  to  characterize 
their  role  without  undertaking  a  systems-level,  mathematical  analysis.  In  particular,  the 
FLETE  theory  describes  three  functions  that  can  be  simultaneously  served  by  Renshaw 
cells.  They  help  stabilize  the  stretch  reflex  by  upper-bounding  its  gain.  They  compensate 
for  non-linearities  introduced  by  the  size  principle  of  motoneuron  recruitment,  which  is  a 
fundamental  principle  about  how  cells  with  multiple  scales  cooperate  to  achieve  smooth 
force  generation  in  all  muscle-like  actuation  systems.  Third,  Renshaw  cells  provide  a 
node  that  works  as  a  hook  or  port  into  the  spinal  servo  circuit.  Through  this  port,  signals 
from  the  higher  nervous  system  can  rapidly  switch  the  system  between  two  distinct 
modes:  a  postural  mode  and  a  rapid/ forceful  movement  mode. 

In  a  1997  study  (#  43)  we  developed  a  model  of  general-purpose  adaptive  control  of  such 
mode  switching.  This  work  combined  VITE,  FLETE  and  a  cerebellar  circuit  into  a 
unified  model  for  adaptive  improvement  of  the  performance  of  voluntary  arm 
movements.  Many  movements  involve  force  generation  by  multiple  muscles  acting  at 
one  or  several  joints.  In  most  advanced  vertebrates,  and  in  all  mammals,  proper 
coordination  of  force  generation  by  multiple  muscles  is  a  learned  skill  that  is  partly 
mediated  by  the  cerebellum.  A  key  goal  of  our  work  on  a  composite  VITE-FLETE- 
Cerebellum  model  has  been  to  understand  exactly  how  cerebellar  learning  can  greatly 
improve  upon  the  performance  that  is  possible  from  the  combination  of  a  kinematic  CPG 
circuit  (VITE)  with  a  feedback-guided  force-generation  circuit  (FLETE).  This  project  led 
to  several  results  of  great  interest  for  robotics.  It  shows  how  joint  receptors  can  act  in 
concert  with  kinematic  error  and  force  receptors  to  achieve  the  independent  control  of 
limb  position  and  stiffness  in  systems  using  both  mono-articular  and  bi-articular  muscles. 
It  also  shows  how  the  cerebellum  can  learn  to  use  the  Renshaw  cell  spinal  circuit  to 
switch  the  spinal  servo  between  a  postural  mode  and  a  rapid  movement  or  high  force 
mode  in  a  task-appropriate  way.  The  result  is  a  self-organizing  system  with  a  highly 
stable  default  mode  and  the  ability  to  use  learning  to  safely  switch  to  a  higher 
performance  mode  whenever  conditions  warrant. 

3.  The  Direction  to  Rotations  Effector  Control  Transform  (“DIRECT”)  model  for 
inverse  kinematic  control  of  an  arm  with  redundant  degrees  of  freedom.  Relevant 
references:  5,  6, 28, 29, 44, 45, 69,  75, 76,  77. 

The  initial  version  of  the  DIRECT  model  was  developed  in  1992-93  to  propose  a  solution 
for  a  fundamental  general  problem  that  is  often  called  the  Motor  Equivalence  Problem: 
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How  can  an  animal  or  robot  learn  to  use  the  mechanical  degrees  of  freedom  of  its  body  to 
accurately  control  spatially  directed  movements  of  its  end-effectors,  with  or  without 
visual  feedback?  For  example,  how  can  a  primate  learn  to  coordinate  movements  of 
back,  shoulder,  upper  arm,  lower  arm,  wrist,  and  fingers  in  order  to  make  a  straight-line 
hand  movement  in  3-D  space,  in  a  desired  direction  or  to  a  desired  endpoint?  In  addition, 
how  can  such  a  system  use  a  novel  tool,  or  adapt  to  novel  constraints  on  moving  joints, 
on  the  fly,  without  making  movement  errors?  This  is  an  ill-posed  problem,  because  there 
are  more  degrees  of  freedom  to  control  than  spatial  dimensions.  It  is  a  difficult  problem 
because  the  motion  component  imparted  to  the  hand  by  motion  at  any  contributing  joint 
depends  on  the  angles  of  all  the  other  joints,  i.e.,  on  the  current  posture  or  joint 
configuration.  Also,  the  functional  relations  depend  on  limb  segment  sizes,  which  can 
grow  continuously  after  birth.  The  model’s  solution  is  to  use  exploratory  movements  of 
the  arm  to  incrementally  discover  and  learn  a  posture-dependent  mapping  from  directions 
in  3-D  space  to  directions  (rotation  vectors)  in  the  higher  dimensional  space  defined  by 
the  angles  of  the  contributing  joints.  Once  this  mapping  has  formed  within  an  adaptive 
neural  network,  it  can  be  used  on  the  fly  to  automatically  recruit  the  joint  rotations 
needed  to  produce  arbitrarily  directed  motions  from  arbitrary  initial  postures.  This  is 
done  directly  and  in  real  time  via  a  neural  network,  without  recourse  to  any  explicit 
representation  of  the  geometry  of  the  system.  Moreover,  the  same  learned  mapping  can 
be  used  to  control  a  tool  held  in  the  hand  without  additional  learning.  Simulation  studies 
have  demonstrated  the  superiority  of  the  DIRECT  model  to  alternative  pseudoinverse 
control  solutions.  In  particular,  DIRECT  remains  well-behaved  near  singularities  (Fiala, 
1995),  and  the  inverse  mapping  can  be  implemented  in  a  way  (Cameron,  1996)  that  is 
very  economical  from  the  perspective  of  computational  degrees  of  freedom  (number  of 
cells  in  the  intermediate  layer  of  the  network  that  computes  the  pseudoinverse 
transformation).  Several  applications  in  robotics  have  been  explored  by  the  laboratory  of 
Professor  Juan  Lopez  Coronado  in  Spain  (e.g.,  #  77). 

DIRECT  model  principles  have  been  incorporated  into  the  DIVA  model  of  speech 
production.  The  DIVA  model  has  been  developed  by  Professor  Frank  Guenther  and  his 
students  (Guenther,  1992  etc.).  It  explains  how  the  speech  articulators  can  be  efficiently 
recruited  to  produce  desired  sounds  from  arbitrary  initial  postures  of  the  speech 
apparatus.  In  both  the  speech  and  limb  motor  control  applications,  the  DIRECT/DIVA 
models  reproduce  the  Motor  Equivalence  property  that  humans  exhibit:  the  ability  to 
automatically  recruit  alternative  motor  means  when  blockages  or  unusual  initial 
conditions  make  more  typical  means  unavailable.  Note  that  this  is  accomplished  in  a 
continuous  way  by  virtue  of  correct  control  circuit  design,  not  by  preprogramming  a 
response  to  all  anticipated  contingencies. 

The  DIRECT  model  led  us  to  propose  new  methods  for  analyzing  neurophysiological 
data  to  better  characterize  the  coordinate  systems  used  to  represent  movement  commands 
in  the  cerebral  cortex.  The  initial  mathematical  analysis  (Ajemian,  Bullock  & 

Grossberg,  1998)  has  shown  how  an  analysis  of  movement  vector  field  structure  can 
differentiate  between  coordinate  systems  that  would  be  difficult  to  distinguish  without 
this  method.  The  results  support  earlier  VITE  hypotheses  about  the  role  of  motor 
coordinates,  and  motor  cortex,  in  trajectory  generation. 
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4.  Adaptive  timing  models  of  the  cerebellum.  Principal  references:  16,17,18,19,21, 
26, 27, 43, 45, 47, 53,  62,  70. 

Experimentalists  have  used  the  conditioned  eye  blink  in  mammals  such  as  rabbits  to 
localize  our  general  ability  to  adaptively  time  anticipatory  behaviors.  Adaptively  timed 
release  of  learned  behavior  has  been  localized  in  the  cerebellar  cortex.  But  the  neural 
mechanisms  of  adaptive  timing  in  the  cerebellar  cortex  remained  controversial.  Our  work 
on  this  problem  developed  in  several  stages  from  earlier  work  by  Grossberg,  with  his 
colleagues  John  Merrill  and  Nestor  Schmajuk,  on  similar  adaptive  timing  mechanisms  in 
the  hippocampus.  Hippocampal  and  cerebellar  circuits  cooperate  to  maintain  motivated 
attention  upon  targets  of  interest  for  a  task-appropriate  duration,  while  adaptively  timed 
movements  are  elicited  towards  them.  Our  1996  J.  Neuroscience  article  with  John  Fiala 
is  perhaps  the  first  article  to  link  a  specific  biochemical  mechanism  of  adaptively  timed 
learning  to  the  anatomy,  neurophysiology,  and  behavior  that  it  controls.  In  particular,  we 
proposed  a  critical  role  for  the  metabotropic  glutamate  receptor  system  in  adaptively 
timed  learning.  Two  experimental  articles  published  last  year  in  Nature  provided  strong 
confirmatory  support  for  our  prediction. 

When  this  result  is  combined  with  other  observations  regarding  the  computational 
properties  of  the  cerebellar  cortex,  a  theory  emerges  of  a  neural  machine  that  performs 
arbitrary  self-programming  of  movement  production  systems.  It  transforms  and  combs 
streaming  context  data  to  discover  leading  indicators  of  the  need  to  perform  control 
actions  so  as  to  minimize  a  cost  function.  Then  it  takes  those  actions  at  the  optimal  time 
within  an  interval  of  90ms  to  4  s  post-indicator.  This  explains  why  the  cerebellum  is  so 
critical  to  the  successful  adaptation  of  land  animals  to  a  full  range  of  environmental 
niches,  each  requiring  a  different  motor  repertoire.  The  closest  technological  equivalent 
to  the  cerebellum  would  be  a  self-organizing  production  system,  but  such  systems 
typically  lack  the  critical  adaptive  timing  competence  that  enables  cerebellar  production 
systems  to  exert  real  time  control  of  motor  systems  with  unique  and  variable  dynamics  in 
environments  that  impose  variable  timing  constraints. 

5.  The  VITE WRITE  model  of  cursive  handwriting  production.  Relevant  references:  6, 
22,23,24,28,30,31,55,79. 

This  project  designed  a  neural  controller,  based  on  the  VITE  model,  for  generating 
handwriting  at  variable  speeds,  sizes,  and  styles.  The  VITEWRITE  model  is  a 
hierarchically  organized  architecture  that  addresses  a  number  of  key  issues  concerning 
the  skilled  performance  of  sequential  actions:  What  is  a  motor  program?  How  can  a 
complex  movement  be  flexibly  performed  at  will  with  variable  speed,  size,  and  style 
without  requiring  new  learning?  How  does  the  brain  control  a  redundant  manipulator  that 
possesses  more  degrees  of  freedom  than  the  space  in  which  it  moves?  How  can  smooth 
curvilinear  movements  be  organized  by  such  a  redundant  manipulator?  In  particular,  how 
is  the  timed  launching  of  different  groups,  or  synergies,  of  muscles  achieved  so  that  the 
desired  directions,  distances,  and  curvatures  of  movement  are  achieved?  How  do 
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volitional  "acts  of  will"  that  vary  the  speed  and  size  of  movements  change  distances  and 
curvatures  of  movement  without  changing  global  movement  form  through  time? 

The  VITEWRITE  model  introduces  a  new  concept  of  how  a  "motor  program"  can  control 
sequential  movements.  This  motor  program  is  not  explicitly  represented  in  the  model. 
Rather  it  is  an  emergent  property  of  feedback  between  a  working  memory  representation 
of  desired  movement  directions  and  a  VITE  trajectory  generator.  The  VITEWRITE 
model  shows  how  the  use  of  a  redundant  manipulator  can  simplify  motor  planning  by 
enabling  temporally  unimodal  commands  to  move  different  motor  synergies.  It  also 
suggests  how  an  outflow  representation  of  movement  speed  can  be  used  to  trigger  the 
next  synergy  to  be  performed,  even  while  some  synergies  are  in  mid-flight. 

The  VITEWRITE  model  illustrates  a  new  approach  to  applications  that  call  for  on-the-fly 
scaling  and  composition  of  trajectories.  It  has  also  served  as  a  basis  for  the  study  of 
handwriting  in  Parkinson's  patients  (Contreras-Vidal  et  al.,  1997).  Such  patients  often 
exhibit  small-sized  handwriting,  or  micrographia,  and  a  full  understanding  of  its  genesis 
through  biology-based  models  like  VITEWRITE  should  lead  to  improved  understanding 
of  the  basal  ganglia  and  to  the  development  of  improved  therapeutic  interventions. 

6.  The  viaVITE  model  for  performance  of  viapoint  movements  under  guidance  by  a 
working  memory  that  stores  novel  target  sequences.  Relevant  references:  15, 21, 22, 
23,30,31,78,79. 

Much  of  human  movement  sequencing  is  controlled  from  working  memory,  which  can 
store  novel  sequences  that  are  used  once  and  then  discarded.  An  experimental  task  that 
has  probed  this  ability  is  the  viapoint  movement  task.  Viapoint  (VP)  movements  are 
movements  to  a  desired  point  that  are  constrained  to  pass  through  an  intermediate  point. 
Studies  have  shown  that  VP  movements  possess  properties,  such  as  smooth  curvature 
around  the  VP,  that  are  not  explicable  by  treating  VP  movements  as  strict  concatenations 
of  simpler  point-to-point  (PTP)  movements.  Such  properties  have  led  some  theorists  to 
propose  whole-trajectory  optimization  models,  which  imply  that  the  entire  trajectory  is 
pre-computed  before  movement  initiation.  We  conducted  new  experiments  to 
systematically  compare  VP  with  PTP  trajectories.  Analyses  revealed  a  statistically 
significant  early  directional  deviation  in  VP  movements  but  no  associated  curvature 
change.  An  explanation  of  this  effect  is  offered  by  extending  the  Vector-Integration-To- 
Endpoint  (VITE)  model  (Bullock  and  Grossberg,  1988).  The  model  explains  the  observed 
trajectories  of  VP  and  PTP  movements  as  emergent  properties  of  a  dynamical  system  that 
does  not  pre-compute  entire  trajectories  before  movement  initiation.  The  new  model 
includes  a  working  memory  and  a  stage  sensitive  to  time-to-contact  information.  These 
cooperate  to  control  serial  performance.  The  structural  and  functional  relationships 
proposed  in  the  model  are  consistent  with  available  data  on  forebrain  physiology  and 
anatomy.  In  particular,  the  model  suggests  that  the  basic  double-inhibitory  circuit  of  the 
dorsal  basal  ganglia  may  be  the  minimal  circuit  solution  to  general  problems  inherent  in 
the  preparation  and  serial  release  of  movement  sequences. 


6 


7.  An  oscillatory  VITE  model  for  legato  articulation  in  piano  performance.  Relevant 
references:  22, 23, 78. 

This  project  investigates  the  neural  mechanisms  of  rhythmic  performance  by  modeling 
psychophysical  data  about  piano  performance.  The  brain  contains  many  oscillatory 
movement  generators.  Locomotor  generators  in  the  spinal  cord  exist  for  multi-limb 
coordination.  However,  the  cortical  system  for  discrete  voluntary  movements  is  also 
sometimes  engaged  for  oscillatory  movements.  Good  examples  are  rhythmic  foot 
tapping  or  rhythmic  clapping,  in  which  active  movement  may  be  restricted  to  the  ankle  or 
wrist  joint.  Such  rhythmic  tapping,  or  its  covert  “mental”  residual,  appears  to  be  used  as 
a  timing  device  in  a  wide  range  of  skills.  Such  timing  may  be  most  readily  studied  by 
examining  musical  performance.  One  aspect  of  musical  performance  where  such  timing 
is  quite  subtle  is  in  legato  articulation,  where  the  temporal  overlap  between  successive 
actions  is  under  continuous  control. 

The  key  overlap  times  (KOTs)  required  for  legato  articulation  varies  markedly  with 
tempo.  For  scales/arpeggios  performed  at  inter-onset  intervals  (IOIs)  of  100-1000  ms., 
prior  reports  show  an  increasing  but  non-linear  functional  dependence  of  KOT  on  IOI. 
Because  the  major  non-linearity  appears  in  the  long  IOI  (slow  tempo)  region,  the 
dependence  of  KOT  on  IOI  is  not  attributable  to  gross  biomechanical  factors,  such  as 
finger  inertias.  Simulations  show  that  the  dependence  can  arise  from  a  neural  circuit  in 
which  a  predictive  central  process  and  a  slow  sensory  feedback  process  cooperate  to 
control  articulation.  An  oscillating  neural  network  is  first  constructed  as  an  extension  of 
the  vector-integration-to-endpoint  (VITE)  model  for  voluntary  control  of  movement.  The 
resulting  circuit  exhibits  volition-controlled  oscillation  rates.  It  also  affords  predictive 
control  by  continuously  computing  an  internal  estimate  of  the  remaining  “time-to- 
contact”  (TTC)  with  a  targeted  integration  level,  reaching  of  which  triggers  the 
oscillator’s  next  half  cycle.  At  fixed  successive  threshold  values  of  this  estimate  of  time 
remaining  in  the  current  half  cycle,  the  performer  first  launches  keystroke  n+1  and  then 
lifts  keystroke  n.  As  tempo  slows,  the  time  required  to  pass  between  threshold  crossings 
elongates,  and  KOT  increases.  However,  if  performers  used  only  such  a  central  process 
to  control  articulation,  they  would  not  show  the  bend  seen  in  the  slow  tempo  region  of  the 
KOT  vs.  IOI  function.  The  bend  emerges  if  performers  lift  keystroke  n  whenever  they 
cross  the  second  internal  threshold  or  receive  sensory  feedback  from  stroke  n+1, 
whichever  comes  earlier.  Empirical  estimates  of  sensory  feedback  delay  times  are 
consistent  with  this  interpretation. 

8.  Signal  processing  for  navigation:  Incorporation  of  motor  information  in  processing  of 
visual  optic  flow.  Relevant  references:  32,  71. 

A  key  problem  in  sensory-motor  control  in  mobile  systems  is  how  to  remove  components 
of  sensory  inputs  that  represent  reafference,  i.e.,  the  fed-back  effects  of  self-motion.  An 
animal  or  robot  that  cannot  distinguish  between  inputs  caused  by  external  motions  and 
inputs  caused  by  self-motion  will  have  severe  performance  limitations  because  it  will 
react  inappropriately  to  many  motion  signals  that  arise  during  self-motion.  For 
successful  navigation,  two  key  variables  that  need  to  be  reliably  extracted  from  optic  flow 
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are  heading  direction  and  time  to  contact  between  looming  objects  and  the  body.  For 
both  variables,  self-motion  can  lead  to  mis-estimates  if  it  is  not  appropriately 
compensated.  Following  a  strategy  similar  to  that  used  in  the  DIRECT  model,  we  have 
developed  a  model  that  is  capable  of  self-organizing  estimates  of  heading  and  time-to- 
contact  by  using  an  action-perception  cycle  during  which  reliable  correlations  emerge 
between  self-generated  motions  and  resultant  optic  flow  patterns.  This  adaptive  system 
successfully  learns  these  estimates  in  the  absence  of  any  explicit  knowledge  of  sensor 
geometry,  motion  signal  gains,  or  the  proximal  motoric  effects  of  self-motion  commands. 
Thus,  the  same  system  architecture  can  be  used  to  control  different  platforms  that  utilize 
different  motors  and  sensors. 

9.  A  neural  model  of  multimodal  adaptive  saccadic  eye  movement  control  by  superior 
colliculus.  Relevant  references:  1,  52,  53, 74, 76,  80, 81,  85. 

A  critical  question  for  the  efficient  acquisition  of  visual  information  is  how  the  eyes  can 
be  rapidly  moved  to  foveate  the  most  salient  targets  at  any  time.  A  still  more  general 
question  concerns  how  distributed  learning  enables  multiple  sources  of  information  to  be 
used  to  make  context-sensitive  behavioral  choices  that  are  appropriate  at  that  moment. 

The  case  of  saccadic,  or  ballistic,  eye  movements  nicely  illustrates  the  new  concepts, 
organizational  principles,  and  neural  mechanisms  that  are  needed  to  explain  such  a 
competence.  It  is  known  that  multiple  brain  learning  sites  are  needed  to  calibrate  the 
accuracy  of  saccadic  eye  movements.  This  is  true  because  saccades  can  be  made 
reactively  to  visual  cues,  attentively  to  visual  or  auditory  cues,  or  planned  in  response  to 
memory  cues  using  visual,  parietal,  and  prefrontal  cortex,  as  well  as  superior  colliculus, 
cerebellum,  and  reticular  formation.  The  organization  of  these  sites  has  been  probed  by  a 
variety  of  saccadic  adaptation  paradigms,  in  which  incomplete  and  asymmetric  transfer  of 
learning  occurs  between  different  tasks.  A  neural  model  of  saccadic  system  organization, 
notably  its  distributed  learning  sites,  has  been  developed  to  explain  these  data.  In 
addition,  it  is  known  that  multiple  modalities  send  converging  signals  to  the  superior 
colliculus  to  determine  which  target  will  be  selected  for  attentional  focusing  and 
movement.  A  neural  model  of  how  these  multiple  inputs  from  visual  cortex,  parietal 
cortex,  and  prefrontal  cortex  are  fused  through  multi-modal  learning  into  a  self-consistent 
map  for  movement  selection  has  also  been  developed.  This  model  has  been  used  to 
quantitatively  simulate  neurophysiological  data  about  the  dynamics  of  burst  and  buildup 
cells  in  the  deeper  layers  of  the  superior  colliculus  during  a  variety  of  movement  tasks. 
Finally,  a  model  of  how  the  saccadic  system  converts  these  signals  from  distributed 
decisions  into  properly  calibrated  movements  has  also  been  developed.  This  model 
explains  the  functional  role  of  the  cell  types  in  the  saccade  generator  in  the  reticular 
formation  that  are  used  to  generate  saccadic  eye  movements,  and  simulates  heretofore 
unexplained  neurophysiological  and  behavioral  data  about  saccades. 

The  first  study  in  this  series  analysed  how  the  saccadic  movement  system  can  select  a 
target  when  visual,  auditory,  and  planned  movement  commands  differ.  In  particular, 
visual  information  is  received  in  retinotopic  coordinates,  and  auditory  information  in 
head-center  coordinates.  Both  types  of  information  need  to  be  transformed  into  motor 
error  coordinates  in  order  to  make  a  saccade  with  the  correct  direction  and  length.  In 
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addition,  the  brain  needs  to  be  able  to  decide  whether  to  reactively  look  at  targets  which 
are  defined  by  momentary  sensory  cues,  or  to  look  towards  targets  which  are  part  of  an 
action  plan.  The  model  suggests  how  visual  and  planned  target  positions  become  aligned 
through  learning  with  visually  reactive  target  positions  to  select  a  movement  command. 
For  this  to  occur,  the  retinotopic  visual  target  representation  is  easily  converted  into  a 
map  of  motor  error  coordinates.  The  auditory  and  planned  head-centered  representations 
of  target  position  are  transformed  into  this  motor  error  map  through  multimodal  learning. 
This  multimodal  fusion  of  information  occurs  in  the  deeper  layers  of  the  superior 
colliculus.  The  model  shows  how  competition  among  these  multiple  sources  of 
information  can  focus  attention  upon,  and  thereby  select,  the  most  salient  target  at  any 
time.  The  mechanisms  that  focus  attention  upon  salient  targets  also  help  to  stabilize 
multimodal  map  learning  through  time.  These  mechanisms  are  variants  of  Adaptive 
Resonance  Theory,  or  ART,  circuits.  Hence  the  model  is  called  the  SACCART  model.  In 
summary,  the  model  suggests  how  multimodal  learning,  attention,  and  movement 
selection  can  be  achieved  by  a  single  circuit.  The  model  hereby  provides  a  new 
framework  for  multimodal  fusion  of  information  that  is  coded  in  different  coordinate 
systems  by  a  system  that  can  stabilize  its  own  learning  while  attentively  selecting  the 
most  salient  piece  of  information  at  any  time. 

This  map  learning  process  enables  the  model  to  quantitatively  simulate  challenging 
recent  neurophysiological  and  motor  psychophysical  data  about  saccadic  movements.  In 
particular,  recent  data  about  superior  colliculus  reveal  a  spreading  wave  of  activation 
across  buildup  cells.  The  peak  activity  of  this  wave  tends  to  occur  at  the  map  location  that 
codes  the  current  gaze  error.  In  contrast,  the  map  location  of  peak  activity  remains 
constant  in  a  separate  population  of  burst  cells,  whose  activity  levels  decay  with  residual 
gaze  error.  The  SACCART  model  quantitatively  simulates  neurophysiological  data  about 
burst  and  buildup  cells,  during  a  variety  of  movement  tasks,  as  a  manifestation  of  the 
mechanisms  that  transform  head-centered  target  representations  into  motor  error 
coordinates.  The  model  also  simulates  neurophysiolgical  data  about  multimodal 
enhancement  and  suppression  of  matched  and  mismatched  target  information  in  the 
superior  colliculus,  and  predicts  a  role  for  known  NMDA  receptors  in  the  multimodal 
movement  map  learning  process. 

10.  A  neural  model  of  the  saccade  generator  in  the  reticular  formation.  Relevant 
references:  1, 52, 53,  74,  76,  80,  81,  85. 

How  are  the  commands  that  are  generated  in  the  motor  error  map  of  the  superior 
colliculus  used  to  move  the  eye.  In  particular,  how  is  this  SPATIAL  information 
converted  into  a  properly  calibrated  TEMPORAL  code  for  the  production  of  a  saccade? 
From  the  early  1970’s  there  has  been  a  long  succession  of  local  circuit  models  of  the 
saccade  generator  in  the  brainstem,  starting  with  the  classical  negative  feedback  model  of 
Robinson  (1975).  All  of  these  models  have  failed  to  explain  key  saccadic  data  and  have 
made  hypotheses  that  have  not  been  supported  by  subsequent  experiments.  A  new 
FOVEATE  model  shows  how  Feedback  Opponent  VEctor  ARchiTEcture  mechanisms 
can  explain  a  much  larger  data  set  while  using  only  well-known  cell  types  in  the  reticular 
formation  as  a  basis  for  the  model  circuit.  The  critical  new  idea'is  how  to  yoke  together  a 
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disinhibitory  feedforward  circuit  with  a  negative  feedback  circuit.  This  yoking  of  circuits 
leads  to  a  self-reseting  trajectory  controller  that  has  many  desirable  properties.  From  the 
robotics  perspective,  the  model  suggests  how  to  robustly  coordinate  horizontal  and 
vertical  movement  controllers  for  active  vision  systems.  In  particular,  the  model  can 
generate  smooth,  synchronous,  straight  trajectories  despite  different  component  motion 
amplitudes.  It  also  provides  a  good  interface  for  inputs  from  both  reactive  and  goal- 
directed  generators  of  visual  tracking  movements. 

1 1 .  A  neural  model  explains  how  distributed  learning  achieves  task-appropriate 
adaptation  of  eye  movement  control.  Relevant  references:  1,  52, 53,  74, 76,  80, 81, 
85. 

Many  parts  of  the  brain  compute  possible  targets  for  eye  movements,  and  multiple  brain 
learning  sites  are  needed  to  calibrate  the  accuracy  of  these  movements.  This  is  true 
because  saccades  can  be  made  reactively  to  visual  cues,  attentively  to  visual  or  auditory 
cues,  or  planned  in  response  to  memory  cues.  Different  parts  of  the  brain  are  used  to 
compute  these  different  types  of  information,  including  visual,  partietal,  and  prefrontal 
cortex,  as  well  as  superior  colliculus,  reticular  formation,  and  the  cerebellum.  A  key 
problem  for  the  saccadic  eye  movement  system— indeed  for  any  distributed  learning 
system— is  how  to  guarantee  that  different  combinations  of  brain  regions  can  all  generate 
correctly  calibrated  movement  commands.  This  requires  a  suitably  designed  distributed 
learning  system. 

The  organization  of  these  multiple  learning  sites  has  been  probed  by  displacing  a  visual 
target  during  a  saccade.  Multiple  task  conditions  have  been  devised  to  probe  different 
aspects  of  the  learning  system,  including  electric,  step,  overlap,  scanning,  and  memory 
paradigms.  Psychophysical  experiments  have  demonstrated  different  patterns  of 
incomplete  and  asymmetric  transfer  of  learning  between  pairs  of  these  paradigms.  A 
neural  model  of  the  saccadic  system  was  developed  to  explain  how  these  distributed 
learning  sites  are  organized  to  correctly  calibrate  saccades  during  multiple  saccadic 
conditions,  and  thereby  to  explain  the  learning  transfer  data,  as  well  as  data  about 
saccadic  coordinate  changes. 

This  model  shows  how  a  combination  of  map  learning  in  the  superior  colliculus  and  the 
prefrontal  cortex,  and  gain  learning  across  multiple  pathways  in  the  cerebellum  can 
achieve  the  desired  results.  In  particular,  the  reactive  pathway  includes  the  superior 
colliculus,  the  attentive  pathway  includes  the  posterior  parietal  cortex,  and  the  planned 
pathway  includes  the  prefrontal  cortex  and  the  frontal  eye  fields.  This  model  provides  a 
prototypical  example  of  how  distributed  learning  can  allow  a  system  with  multiple  bases 
for  movement  generation  to  maintain  and  recover  accuracy  despite  significant  changes  in 
internal  parameters,  including  damage  to  one  or  another  central  movement  generator. 
Multiple  bases  for  movement  generation  are  a  necessity  in  any  freely  moving  system  that 
must  pursue  goals  while  also  reacting  to  unscheduled  opportunities  or  challenges. 

12.  Neural  control  of  interlimb  oscillations:  Human  bimanual  coordination,  and  biped 
and  quadruped  gait  control  and  bifurcations.  References:  37, 38,  39, 40,  73,  80, 82. 
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The  previous  projects  have  analysed  phasic  movements  of  the  arms  and  eyes,  as  well  as  a 
role  for  rhythmic  control  of  sequential  movements.  The  present  projects  were  devoted  to 
modeling  the  central  pattern  generators  whereby  humans  and  animals  can  accomplish 
oscillatory  movements.  These  results  clarify  how  both  phasic  and  oscillatory  movements 
can  be  controlled  by  the  same  limb.  It  was  shown  how  the  ubiquitous  on-center  off- 
surround  feedback  circuits  that  exist  throughout  the  brain  could  be  specialized  for  this 
task.  In  particular,  the  excitatory  neurons  in  these  circuits  operate  on  a  faster  time  scale 
than  their  inhibitory  circuits.  A  descending  GO  signal  command  -  as  in  the  VITE  model 
-  activates  movement  gaits  and  triggers  gait  transitions  as  its  amplitude  increases.  A 
single  model  central  pattern  generator  can  elicit  both  in-phase  and  anti-phase  oscillatory 
movements  at  different  GO  amplitudes.  All  the  gaits  and  their  transitions  are  emergent 
properties  of  this  central  pattern  generator. 

Simulations  of  human  bimanual  coordination  tasks  show  how:  anti-phase  oscillatory 
movements  of  the  two  hands  at  low  frequencies  switch  to  in-phase  oscillations  at  high 
frequencies;  in-phase  oscillations  occur  at  both  low  and  high  frequencies;  phase 
fluctuations  occur  at  the  anti-phase  in-phase  transition;  a  "seagull  effect"  of  larger  errors 
occurs  at  intermediate  phases;  and  oscillations  slip  toward  in-phase  and  anti-phase 
when  driven  at  intermediate  phases. 

Simulations  of  quadruped  vertebrate  gaits  include  the  amble,  the  walk,  all  three  pairwise 
gaits  (trot,  pace,  and  gallop),  and  the  pronk.  Rapid  gait  transitions  have  been  simulated  in 
the  order  -  walk,  trot,  pace,  and  gallop  -  that  occurs  in  the  cat,  along  with  the  observed 
increase  in  oscillation  frequency.  Precise  control  of  quadruped  gait  switching  uses  GO- 
dependent  modulation  of  inhibitory  interactions,  which  generates  a  different  functional 
anatomy  at  different  arousal  levels,  a  design  property  that  has  been  demonstrated  even  for 
invertebrates.  The  primary  human  gaits  (the  walk  and  the  run)  and  elephant  gaits  (the 
amble  and  the  walk)  are  simulated,  without  such  modulation,  by  oscillations  with  the 
same  phase  relationships  but  different  waveform  shapes  at  different  GO  signal  levels, 
much  as  the  duty  cycles  of  the  feet  are  longer  in  the  walk  than  in  the  run.  Relevant  neural 
data  from  spinal  cord,  globus  pallidus,  and  motor  cortex,  among  other  structures,  are  also 
clarified. 

Summary 

Human  competence  will  remain  the  model  for  intelligent  robotics  into  the  indefinite 
future.  Limitations  of  current  robotic  deployments  can  only  be  overcome  if  robots 
acquire  the  adaptivity,  robustness,  and  flexibility  needed  to  allow  them  to  compose  goal- 
directed  behavior  on  the  fly.  Our  studies  have  been  explicating  the  neural  circuit 
architectures  that  make  humans  able  to  quickly  compose  adequate  actions  adapted  to 
novel  momentary  requirements,  and  then  to  perfect  their  execution  of  those  actions  when 
conditions  allow  their  repeated  use.  This  amounts  to  experience-guided  self¬ 
programming  or  “self-organization”  -  the  only  truly  general  and  practical  alternative  to 
the  impossible  task  of  attempting  to  pre-program  a  response  to  every  possible 
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contingency.  The  same  self-organizing  abilities  allow  recovery  after  damage,  so 
machines  based  on  these  ideas  will  also  be  self-repairing. 

As  a  result  of  using  mathematics  to  explicate  the  neural  circuit  bases  of  human  sensory- 
motor  flexibility,  we  have  been  able  to  construct  and  simulate  several  of  the  most 
comprehensive  neural  system  models  ever  seen.  These  models  have  consistently  raised 
the  bar  in  the  field  of  computational  neuroscience,  and  we  expect  them  (and  their 
descendents)  to  serve  as  prototypes  for  many  robotic  designs  going  forward.  Although 
many  of  the  central  circuits  are  already  being  applied,  this  process  will  further  accelerate 
when  technology  begins  to  catch  up  with  biology  in  key  areas,  such  as  miniature 
embedded  sensors  and  practical  muscle-like  actuation. 
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